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Abstract. One of the main notions of information theory is the notion of mutual information 
in two messages (two random variables in Shannon information theory or two binary strings in 
algorithmic information theory). The mutual information in x and y measures how much the 
transmission of x can be simplified if both the sender and the recipient know y in advance. Gacs and 
Korner gave an example where mutual information cannot be presented as common information 
(a third message easily extractable from both x and y). Then this question was studied in the 
framework of algorithmic information theory by An. Muchnik and A. Romashchenko who found 
many other examples of this type. K. Makarychev and Yu. Makarychev found a new proof of Gacs- 
Korner results by means of conditionally independent random variables. The question about the 
difference between mutual and common information can be studied quantitatively: for a given x 
and y we look for three messages a, b, c such that a and c are enough to reconstruct x, while b and 
c are enough to reconstruct y. 
In this paper: 

— We state and prove (using hypercontractivity of product spaces) a quantitative version of 
Gacs-Korner theorem; 

— We study the tradeoff between \a\ ,\b\ , |c| for a random pair (x, y) such that Hamming distance 
between x and y is en (our bounds are almost tight); 

— We construct "the worst possible" distribution on (x, y) in terms of the tradeoff between 
H,\b\,\c\. 

1 Introduction 

Let us start with a specific communication problem. Say we have two n-bit strings x and y 
such that Hamming distance between them is exactly en. We want to deliver x and y to the 
corresponding network nodes (Fig. [I]). For that we send a common message c to both nodes, 

a 



c 



b 



Fig. 1. Network information transmission problem 

and separate messages a and b to each node. What tradeoff between the lengths of a, b and c is 
possible (if we are willing to be able to transmit any such pair (x, y))? If we require c to be empty, 
then, clearly, \a\ , |6| > n. On the other hand, if a and b are empty, then |c| > (1 + H (e) + o(l))n. 
In this paper we study this tradeoff and obtain almost tight bounds for \a\, \b\ and |c|. 

We can consider more general situation. Suppose we have two dependent random variables 
X and Y. We sample n independent copies of (X, Y) (let us call them (A^Y 1 ), (X 2 ,Y 2 ), 
. . . , (X n , Y n )) and want to transmit x := X 1 X 2 . . . X n and y := Y l Y 2 . . . Y n . We want our 
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transmission to be successful with probability close to 1. Again we are interested in the tradeoff 
between |a|, \b\ and |c|. Information Theory gives three trivial bounds: 



This transmission problem was first considered in [T] . In the paper [2J Gacs and Korner showed 
that the first two bounds could not be tight simultaneously (that is, "common information 
is less than mutual information") unless the joint distribution of X and Y is in some sense 
degenerate. Konstantin Makarychev and Yury Makarychev presented in [3] another proof of the 
same theorem using a notion of conditional independence of random variables. 

One can also pose a similar problem in the framework of Algorithmic Information Theory. 
Suppose we have two strings x,y G {0, l} n such that C(x) w n, C(y) ps n and I(x : y) 
0.5n (here C(-) stands for plain Kolmogorov complexity and /(• : •) — for algorithmic mutual 
information). Does there exist a string z such that 

- C{z) < 0.51n, 

- C(x | z) < 0.51n, 

- C(y | z) < 0.51n? 

Again, it turns out that the answer is negative in general (see [1], [5], [6], [7]). Information- 
theoretic and Kolmogorov-complexity-theoretic formulations are similar: a stands for the short- 
est program, which converts z to x, b stands for the shortest program, which converts z to y, 
and c stands for the shortest program, which outputs z. 

All these problems can be reformulated using a simple unifying combinatorial notion of 
the profile of a bipartite graph. Suppose we have a bipartite graph E C X x Y. Consider the 
following communication problem: transmit endpoints of an edge of E. 

Definition 1. We say that a triple (a,/3,7) belongs to the profile of E if there exist two map- 
pings f : {0, l} a x {0, l} 7 -> X, g : {0, l}' 3 x {0, l} 7 -»■ Y such that for every (x, y) G E there 
exist a G {0, 1} Q , b G {0, 1} , c G {0, l} 7 such that x = /(a, c) and y = g(b, c). 

There is an obvious one-to-one correspondence between the profile of E and the set of possible 
lengths of three messages that we may transmit if we are willing to be able to deliver endpoints 
of any edge in E. We can consider profiles not only for graphs, but also for distributions (we 
require the existence of a, b, c with high probability). If we consider the adjacency matrix M of 
E, then the abovementioned question can be reformulated as follows: for which triples (a, /3, 7) 
it is possible to cover all ones in M with 2 7 combinatorial rectangles of size 2 a x 

In this paper we study profiles of various graphs and distributions. In Section [2] we relate 
the profile of E and the maximum number of ones in the rectangles in M of a given size. In 
one direction the relation is obvious, but suprisingly it turns out that in some cases (namely, 
for edge-transitive graphs) the existence of a rectangle with many ones implies the existence 
of a good rectangle cover. We also prove several simple bounds on the profile of a bipartite 
graph which will be useful later. In Section [3] we devise a pretty general tool to upper-bound 
number of ones in rectangles in M of a given size. It relies on hypercontractivity in product 
spaces (in [8] hypercontractivity was used for a similar information problem). Using it we prove 
a quantitative version of Gacs-Korner Theorem (for the case of uniform marginal distributions) 
and improve upon [2] and 0. In Sections [| and [5] we prove almost tight bounds for the profile 
of our first example (strings with Hamming distance en) using all these tools. Then in Section 
[6] we build an example of a graph with a minimal possible profile (unsurprisingly, we just prove 
that random graphs do the job). This example was announced in [7J without proof. 



\a\ + \c\ > (H(X) + o(l))n, 
\b\ + \c\ > (H(Y) + o(l))n, 
o| + |6| + |c| > (H(X,Y) + o(l))n. 



(1) 
(2) 
(3) 
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2 Profile and the maximal number of edges in a rectangle 

In this section we make two observations. Let E C X x Y be a set of edges of a bipartite graph. 
For given a and (3 consider combinatorial rectangles X' x Y' where X' C X has cardinality 
2 a and F'CY has cardinality 2 13 . Let Re(cc,/3) be the maximum number of edges covered by 
rectangle of this type. 

Proposition 1. If Re(cx, (3) • 2 7 < \E\, then the triple (a,/9, 7) does no£ belong to the profile of 
\E\. 

This observation is obvious: each of 2 7 rectangles covers at most Re(oi,/3) edges, so they 
cannot cover the entire graph. 

It turns out that this observation can be reversed for the case of edge-transitive graphs. 
Recall that a bipartite graph E C X x Y is edge-transitive if the group of its automorphisms 
acts transitively on E (for every two edges e and e' there is an automoprhism that maps e to 
e'; automorphisms are pairs of permutations X — > X and Y — > Y that generate a permutation 
of E). 

Proposition 2. Awme £/teii E is edge-transitive. If Re{ol, (3) ■ 2 7 > \E\, then the triple 
(a, (3, 7 + log log \E\) belongs to the profile. 

Proof. Let R = X' x Y' be a rectangle that contains many edges: let if be the number of edges 
in it, so that K ■ 2 7 > \E\. We will cover E by shifted copies of R. Consider m independent 
randomly chosen automorphisms (all elements of the group are equiprobable) ; let R\ , . . . , R m 
be the images of R under these automorphisms. 

We want to show that R±, . . . , R m cover E with positive probability. Indeed, for a given edge 
e the probability of being covered by one Ri is Kj \ E\ (the preimage of e under the automoprhism 
is uniformly distributed in E due to edge transitivity). The probability of not being covered 
is therefore (1 — Kj \E\). Different automorphisms are independent, so the probability of e to 
avoid all R, L is (1 — K/\E\) m . The probability that some edge is not covered is bounded by 
\E\ • (1 — Kj \E\) rn . The assumption guarantees that Kj \E\ < 2~ 7 , so m = 2 7 log \E\ is enough 
to make this probability less than 1. (Indeed, (1 — 2~ 7 ) 27 « 1/e < 1/2.) 

In the typical application the values of a, (3, 7 are of the same order of magnitude as log \ E\, 
so log log \E\ is small compared to a, [3, 7. 

Now we state and prove several (trivial) bounds on the profile of E. 

Proposition 3. Let E C X xY be a bipartite graph without isolated nodes. Let < a < log \X\ 
and0</3 < log|Y|. 

1. If a + 7 < log |X| or f3 + 7 < log \Y \ , then (a, (3, 7) does not belong to the profile of E . 

2. If a + (3 + 7 < log i/ien (a, /3, 7) does not belong to the profile of E. 

3. If min(a, (3) + 7 > log|i?| ; then (a,/?, 7) belongs to the profile of E. 

4- If a + (3 + 7 > log \X\ + log |Y|, then (a, f3, 7) belongs to the profile of E. 

Proof. 1. If a + 7 < log I X |, then we are unable to cover X (here we use that there are no 
isolated nodes in E). The second case is similar. 

2. If a + (3 + 7 < log then it is obviously impossible to cover all edges. 

3. Using one 2 a x 2^ rectangle we can cover any 2 mm( - a '^ l's. Thus, we can cover E with 
\E\ /2 min ( a ^) rectangles. 

4. If a + f3 + 7 > log |X| + log |Y|, then we can cover not only E, but the entire X xY. 
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3 Upper-bounding Re (ex., f3) 



To apply Proposition [T] to regular graphs, we need a technique to upper-bound Re (a, /?)• Let us 
consider slightly more general situation. Instead of a regular graph E C X x Y let us consider 
a distribution T> over X x Y such that its marginal distributions are uniform. Then the natural 
generalization of Re(ol,P)/ \E\ is the following quantity: 

Rx>(a,(3):= max Pr^ v \„ v [x £ A, y £ B] . 

\A\<2 a v ' y) 
\B\<2? 

Now let us generalize our problem even more. Suppose that 2?'s marginal distributions are 
not necessarily uniform. Let us denote them by T>x and T>y. Suppose we have two sets iCX, 
B C Y such that Pr Xr uV x [x £ A\ < fi, Pi y ~T>y [y € B] < v. We are interested in upper bounds 
on PTf Xi y\^x) [x £ A,y £ B]. There is an obvious bound min{/i, u} for this quantity, but as we 
will see if T> is in some sense non-degenerate, then we can sharpen this bound. From now on we 
assume that supp T>x = X and supp T>y = Y (otherwise, we can reduce either X or Y). 

Let us call T> non- degenerate if supp T> is a connected bipartite graph on (X, Y) and degen- 
erate otherwise. 

To formulate the upper bound, we shall introduce a parameter 5(T>) with the following 
properties: 

— 5 (T>) > iff P is non-degenerate (Theorem IT] below) ; 

— 5(T>i ®T>2) > min{<5(Pi), 5(T>2)} (Theorem [2] below), here V\ ®T>2 is a product distribution 
of T>\ and T>2 

— if 5(T>) > 0, then there is a non-trivial upper bound on Pr^^^x) [x £ A,y £ B] (Theorem [3] 
below) . 

If P's matrix was symmetric, then one could in principle define 5(T>) := 1 — X(T>), where 
X(T>) is the second largest eigenvalue of P's matrix (the largest eigenvalue is clearly 1). Then, 
obviously, all three desired properties are true (the third one is a corollary of Expander Mix- 
ing Lemma [9]). The problem is that Expander Mixing Lemma is too weak for our purposes 
(especially if fj,, v = o(l)), so we need something stronger. 

Now let us define 6(T>). Consider J^x, J~Y — linear spaces of R- valued functions on X and 
Y respectively. Consider the following linear operator Tx> : Ty — > Fx ■ 

(T v f)(x) :=E {X ,^ V [f(y)\x = x']. 

Let us consider standard L p -norms on Tx and Ty. ||/|| p := E [|/| p ] 1//p (here expectation is 
taken over X or Y with respect to T>x, T^y respectively). It is easy to check that T-p is an 
Lp-contraction for all 1 < p < oo. 

Lemma 1. \\T v f\\ p < ||/|| p for all 1 < p < oo and f £ Ty . 

S(V) characterizes to what extent Lemma [JJ can be sharpened for a particular T> near p = 2. 

Definition 2. 

S(V) := max [d < 1 | V/ £ Ty ||r c /|| 2+ ^ < ||/|| 2 _ 5 } 
This definition makes sense because the following Lemma holds. 
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Lemma 2. // 1 < p < q < oo, then 

< 



\p 

there is an equality iff f is constant. 
Now let us restate the properties of 5(V). 
Theorem 1. 5(D) > iffD is non- degenerate. 
Theorem 2. 5(D 1 ®D 2 ) > min {5(D X ), 5(D 2 )}. 

Theorem 3. Let A C X, B C Y. IfPr Xr ^ Vx [x G A] < \i and Pr y ^T> Y [y G B] < v, then 
P*(x, y )~T> [x G A, y G B) < fj,u+ 



For the proof of Theorem [2] see [10]. Theorem [T] and Theorem |3] will be proved in the Appendix. 
We will typically apply Theorem [3] in situations, where [i and v tend to zero. Let us instantiate 
Theorem [3] for this case. 

Corollary 1. If = o(l) and v = o(l) then 

Pr ( ,^ [x G A,y G B] < O (V) 1 ^-^)) 

Now we state and prove a quantitative version of Gacs-Korner theorem [2] for the case of 
uniform marginal distributions. 

Theorem 4. Let D be a distribution over X x Y with uniform marginal distributions. We 
sample n independent copies (X , Y ), (X 2 ,Y 2 ), (X n ,Y n ) of T> and want to transmit 
x := X 1 X 2 ...X n andy := Y X Y 2 . . . Y n as in Fig. (with probability 1 - 2~ n ^ ). Then, 

\a\ + \b\ + (2 - 5(D)) \c\ > (log |X| + log \Y\ + o(l))n. 

That is, if V is non-degenerate, then the bounds ([!]) and Q could not be tight simultaneously, 
since 5(D) > 0, H(X) = log|X|, and H(Y) = \og\Y\. 

The proof is a trivial combination of Theorem [2j Corollary [TJ and Proposition [TJ 



4 Fixed distance graph and its rectangles 

In this section we use Propositions [T] and [2] and the results of Section [3] to analyze the trans- 
mission of two strings with Hamming distance en (see the beginning of Section [TJ . 

Let us consider the following bipartite graph G nj£ C {0, l} n x {0, l} n . There is an edge (x, y) 
iff Hamming distance between x and y is exactly en. 

We are interested in the profile of G U)E . Let us denote M„ i£ its adjacency matrix. For sim- 
plicity we restrict ourselves to the case where a and (3 (lengths of messages a and b sent to each 
node separately) are equal: a = j3 = rn, 7 = xn, where 0<r<l,0<x<l + H(e) are 
constants. Let us denote by A(e, r) the following fraction: 

A{£jT)= Hm log(ie( g ,r,n)/|G w , e |) 

n— >oo n 

where R(e,r,n) is the maximum number of l's in a rectangle of M n ^ e of size 2 Tn x 2 Tn , and 
|G n ,e| is the total number of edges in G„ jE . Since G nj£ is edge-transitive, bounds for A can be 
directly translated into profile bounds (via Propositions [l] and [2| . 
Let us state our main combinatorial result: 
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Theorem 5. (Lower bound) If r < H (l - \/l - e), tfien yl(e,r) > -(1 + H(e) - 2r). 7/ 
T>H(l- yjl-e), then 

(±-\ 



A(s, t) > - 1 + - t - off — - (1 - a)if 



V2a7 v ' V 2 ( 1 -«) 

where < a < 1/2 and -ff(a) = r. 
(Upper bound) A(e, r) < — xEf- 

The lower bound on r) 

To prove that A(e, r) is large we show that some rectangle in M n ^ £ has many l's. Indeed, consider 
a rectangle C xC, where elements of C are strings that contain exactly an ones; obviously there 
are 2 (#(aO+°(l))« of them (i.e. r = if (a) + o(l)). The number of l's in C X C equals 

n \ / an \ /(l - a)n\ _ 2 (ij-( a ) +a _H-( e / 2a )+(l-a)-H"(e/2(l-a!))+o(l))n 

The total number of l's in M n ^ £ is 

2 n f "1 = 2 ( 1 +^( £ )+°( 1 ))™. 

Thus 

^ 2(tf(a)+a#(e/2a)+(i-a)#(e/2(i-a))+o(l))ri 

A(s, t) > lim — • log ,,,„,,, = 

v ' 1 - n^oo n 2( 1 + H ( £ )+°( 1 )) n 

= -(1 + H(e) - H(a) - aH(e/2a) - (1 - a)H(e/2(l - a))). 

If t = H (l — "v/1 — e) , then the sphere gives a rectangle with 2( 2r+ °( 1 )) n ones, which is 
clearly optimal. For r' < r we can subsample this rectangle and get 2 T ' n x 2 T ' n rectangle with 

2 (2r'+o(l))n Qnes 

The upper bound on A(e, r) 

Let us prove that A(e,t) < — jE^- Consider the distribution D n ^ £ on (x,y) G {0, l} n x {0,1}™: 
x is uniformly distributed in {0, l} n ; and y £ {0, 1}™ is obtained from x by independently 
changing each bit with probability e. 

This distribution generates an edge in G n)£ with probability at least 1/ra, and all the edges 
of G nj£ are equiprobable. So instead of counting the number of edges in a rectangle, we may 
estimate the -D n)£ -probability of this rectangle (the factor n does not matter with our precision). 
It is enough to show, therefore, that for every C\, C2 C {0, 1}™ such that |Ci| = | C 2 1 = 2 Tn the 
following inequality holds: 

Pr ( ,^ D _ [(*, y)€dx C 2 ] < 2~^ + °^ n . (4) 

If we show that 5(D n ^ £ ) > 2e, then we can plug this bound into CorollaryJI] and obtain Q. 
Since D Uj£ = -Of 1 ™, using Theorem 2jone can reduce this statement to the following well-known 
inequality. 

Theorem 6 (Two-Point Inequality). 

S(D lt£ ) > 2e. 

For the proof see [10| • 
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5 The profile of the fixed-distance graph 



First, we use Theorem [5] and Propositions [2] and [T] to get explicit bounds for the combinatorial 
profile of the fixed-distance graph G rh£ that improve those given by Proposition [3j 

Theorem 7. Let < r, x < 1 be constants. 

— If x < (1 — t)/(1 — e), then for sufficiently large n the triple (rn, rn, xn) does not belong to 
the profile of G n<e . 

— There are two following "positive bounds". 

• If t < H (l — \/l — e) and x > 1 + H(e) — 2t, then for sufficiently large n the triple 
(rn, m, xn) belongs to the profile of G n>£ . 

• Ifr>H(l- ^T^e) and 

*>! + *(«> -r-„ir (5^), 

where < a < 1/2 cmc? H(a) = r, then for sufficiently large n the triple {rn,Tn, xn) 
belongs to the profile of G n ^ . 

Figure [2] shows the bounds for the profile for e = 0.11 . . . (for this value the total number 
of edges is 2 15n ). It shows trivial upper and lower bounds from Proposition [3J as well as our 
results (Theorem [5]). 

Not that our bounds are tight in two regions: 

— if r < H(l — y/1 — e), then our upper bound is equal to the trivial lower bound; 

— if r = 1 — o(l), then our upper bound is asymptotically equal to our lower bound. 

The same bounds can be obtained for other notions of profile, so the results are directly 
comparable with previous work. Let us show how this can be done for Kolmogorov complexity. 

Let us assume that for every n a bipartite graph E n C {0, l} n x {0, l} n is fixed (and there 

is an algorithm computing E n given n). Let a,/3,7 be some positive rational numbers. Let 

R n (o, (3) be the maximum number of edges covered by a rectangle X' x Y' with \X'\ = 2 an and 
|y'| _ 2^ n 

Proposition 4. There exists some constant d such that: 

— If R n (a, (3) ■ 2 7?1 < \E n \, then for all sufficiently large n for most edges (x,y) £ E n there is 
no string c such that 

C(c) < 7n — dlogn, C(x\c) < an — dlogn, C(y\c) < fin — dlogn. 

— If E n is edge-transitive and R n (a,(3) ■ 2 7 ™ > \E n \, then for all sufficiently large n for every 
edge (x,y) G E n there exists a string c such that 

C(c) < 7n + dlog n, C(x\c) < an + dlogn, C{y\c) < fin + dlogn. 

Proof. For each c we consider a rectangle X' x Y' where X' consists of strings x such that 
C(x\c) < an and Y' consists of strings y such that C(y\c) < fin (\X'\ < 2 an , \Y'\ < 2 l3n ). The 
number of strings c such that C(c) < 7n — dlogn is less than 2 7n /n rf , so the edges covered by 
these rectangles form a minority (l/n rf -fraction); for all other pairs there is no string c with 
described properties. 

The second part of the theorem is also easy. Proposition [2] guarantees that for sufficiently 
large n the set \E n \ can be covered by 2 7ra+ °( logn ) rectangles of size 2 an x 2^ n . By exhaustive 
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-•— trivial lower bound 

-+— trivial upper bound 

-x— lower bound 

-■- upper bound 
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1.0 



Fig. 2. bounds on the profile from Proposition [3] and Theorem[5] d(x, y) = en, e — 0.11 . 



search, we can find first covering of this sort (in some natural order), and this covering is 
determined by n, so its complexity is O(logn). Then we let c be the number of the rectangle 
X' x Y' that covers a given edge (x, y). The complexity of c is at most + O(logn). Knowing 
c (and the entire covering, which has complexity O(logn)) we can describe x and y by their 
numbers in X' and Y' . 

One can also show that a random pair (x, y) generated with distribution D nE will have a 
profile (in terms of complexity) within the bounds from Theorem [4] (with high probability) . 
Indeed, the law of large number says that the number of places where x and y differ is close to 
en, and for each fixed number of difference we get a uniformly random edge in G ns i for e' close 
to e. It remains to note that our bounds are continuous (as functions of e). 

Let us note for comparison that (a weaker) upper bound for this distribution can be obtained 
using conditional independence technique from [6]. It is quite involved, but it also has a form 
x > c(e)(l — r) (where c(-) is an explicit (but cumbersome) function) as in Theorem [Tj So, let 
us compare c(e) with 1/(1 — e) (the bigger value is better) for different values of e (see Fig. [3]). 



e 


1/(1 -e) 


c(s) 


0.1 


1.11. . . 


1.000015... 


0.2 


1.25 


1.016... 


0.3 


1.43. . . 


1.067... 


0.4 


1.67... 


1.33... 



Fig. 3. A comparison of upper bounds from Theorem [5] and from [6] 
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6 A stochastic pair with minimal profile 



A pair with minimal profile is constructed in [7] using the technique developed by An. Much- 
nik [I], [S]. However, this construction is quite artificial and cannot be translated into Shannon 
information theory, because the constructed pair is not a typical object in a simple family (is 
not stochastic in the sense of algorithmic information theory). In this section we show how to 
construct a graph with minimal combinatorial profile (for graphs with given number of edges). 
For that we first prove (using probabilistic arguments) that such a graph exists; after that it 
can be found by brute-force search. This gives us a stochastic pair with minimal profile. This 
result was announced in [7] as an unpublished result of An. Muchnik (1958-2007) and was not 
published since then. 

To analyze random graphs we need a version of Chernoff inequality which deals with nega- 
tively correlated random variables |llj . 

Theorem 8 (Chernoff inequality). Let Xi,X2, ■ ■ ■ ,X n be negatively correlated binary ran- 
dom variables (i.e., for every i\ < i 2 < • • • < ik we have Pr [X^ = 1 A X; l2 = 1 A ... A X{ k = 1] < 
rT) =1 Pr[X ij= l]). 

Let /x = E [Xi + X 2 + . . . + X n ) and 5 > 0. Then 

/ P S 

Pr [Xi + X 2 + . . . + X n > (1 + 8)im] < 



e 



'1 + 5) 



i+s 



Following |7|, we consider (for simplicity) graphs with 2 n vertices in each part and 2 L5n 
edges. We choose a random graph of this type (i.e., uniformly choose a matrix of size 2 n x 2 n 
with 2 L5n ones). 

Theorem 9. // x + r < 1.5 and x + 2r < 2, then for some e > the probability of the event 
"every 2 Tn x 2 rn rectangle in M contains less than 2( L5- ^~ £ ) n ones" is close to 1. 

Proof. Let us choose < e < min(2 — n — 2r, 1.5 — k — r). The entries of M are negatively 
correlated, so the Chernoff inequality (with 1 + 5 = 2( 2 ~ x ~ 2r-£ ) n and /i = 2( 2r-0 - 5 ) n ) guarantees 
that for a fixed rectangle of size 2 Tn x 2 rn the probability of the event "this rectangle contains 
more than 2( 1 - 5 ~ x-£ )™ ones" is bounded by 

, . 9(2r-0.5)n 

I e ^ ^__2( 1 ' 5 -« r - e +°( 1 )) n 



I 2(2-2r->f-e)n2( 2 - 2T -^- £ )" I 

Therefore the probability that some 2 rn x 2 Tn rectangle has too many ones does not exceed 

271 \ 2 

2 TU ) ~ 
Since e < 1.5 — x — r, we are done. 

Remark. Note that it follows from Propositions [T] and [3] that the bounds in Theorem [9] are the 
best possible. 

Another simple observation: with high probability all rows and columns of M contain at 
most 2 • 2 0,5n elements. Indeed, the probability that in a given row (or column) the number of 
ones is twice more than the expected value 2°' 5n , is doubly exponentially small (it follows from 
theorem [8]), and we have only exponentially many rows and columns. 

Then we can follow the plan described above: we conclude that there is a graph with both 
properties and logarithmic complexity; then, using the first part of proposition [4] one can easily 
see that a typical edge (x, y) of this graph has minimal profile in the sense explained in [7\\. (The 
second property is needed to show that the complexities of x and y in a typical edge are close 
to n.) 
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A Properties of 5(T>) 

We prove properties of 5(V) that were promised in Section |3j 
Theorem 10. 5(T>) > iffT> is non-degenerate. 

Proof. Suppose T> is degenerate. Then there are non-trivial partitions X = X\ U X2, Y = 
Y\ U Y<i such that Pr^^^x) \ x £ XuV G ^2] = and Pr^-j^x) [x £ A^,y £ Yi] = 0. Consider 
the indicator Iy x E Ty of Y\ that maps Y\ to 1 and Y — Y\ — to 0. Then for every 1 < p < 00 
the following equality holds: ||Tx>iyi||p = ||iyi||p. On the other hand by Lemma [2] 

-\\Tr,r Yl \\ p >0, 

since supp T>x = X, supp Vy = Y, and Iy x is non-constant. Thus, clearly, S(V) = 0. 

Conversely, let V be non-degenerate. Conside the unit sphere S in L,2(Y). For f E S let us 
define 5(f) := maxj^ < 1 | \\T v f\\ 2+ _s_ < ||/|| 2 -5}- Since 5{V) = ixd feS S(f), 6(f) is continu- 
ous, and S is compact, it remains to prove that 5(f) > for every / E S. If / is constant then, 
the inequality is obvious since is constant. If / is not constant, then the inequality is true, 
since ||Tx>/||2 < 11/ H2, so there exists 77 > such that ||2x>/[|2+»j < H/lh-r? (II • \\p is continuous 
in p). 
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Theorem 11. Let A C X, B C y. IfPr x ^x> x [x £ A] = n and Pr y ^v Y [y <E B] = v, then 
Pr (x, y )~x> [ie^!/e5]< /iz/+ 

+ (^(1 - m> + mi - ^) 1,( " m . (-- i<c »(i - ») + ki - >>r sm ) . 

(5) 

Proof. Let I4 and Ib be indicators of j4 and 1? respectively. 

Let us denote /(x) := /a(^) — M> <7(x) := Ib(%) — v. 

E (cc,y)~z> [-^A(a:)^(y)] = + E (a:,i/)~x> [/faMs/)] = W + E z~x>x [f(x)T v g{x)} 
By Holder's inequality 

Z+ 1-6(T>) 

By definition of <5(2?) 

fJkV+ \\T V f\\ 2+ _s(^_\\g\\2^S(V) < V + \\fh-S(v)\\g\\2-s(v)- (6) 
Let us recall that / = Ia — fJ>, g = Ib — v- Thus, 

- Sm ={^-^ + ^\l-4 K2 ' m \ (7) 

Plugging 0, (|8]| into Q we obtain @. 
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